Iterative Constrained Optimization for Flexible Classifier Design with Multiple Competing Objectives

The design of supervised learning systems for classification require finding a trade-off among a number of competing performance criteria. A common approach is to optimize an overall objective derived from the individual performance objectives. Such a simplified approach promises only for the chosen performance metric when evaluated over the training samples, yet does not make any guarantee on what to happen to the individual objectives. The need for more than a single objective function paves the way for treating the supervised learning problem with multi-objective optimization techniques. The MOP theory offers a great deal of freedom to designers in modeling the design preferences about the individual conflicting objectives. In this paper, we describe a multi-objective programming (MOP) approach to flexible classifier design with many more competing objectives than two. Our formulation of the classifier design problem, which we refer to as iterative constrained optimization (ICO) approach, involves an iterative process of the optimization of individual objectives with proper constraints on the remaining competing objectives. The constraints are obtained from a development set. Moreover, the iterative process is such that the next step is chosen that improves a certain overall metric over a development set. We illustrate the utility of the proposed framework in the context of automatic language identification of 12 languages and 3 dialects, i.e. with 30 objectives. Our experimental results demonstrate that the range of the objective values are greatly reduced compared with what is attained by optimizing an overall objective. We also observe that ICO-trained classifiers attain less outlier objective values than those with conventional SOP-trained classifiers. This is quantified by our experimental results that the false-rejection (FR) rate for Hindi is reduced from 39.47% to 25.00% with only slight changes in others.